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Oscillations of empirical distribution 
functions under dependence® 
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University of Chicago 

Abstract: We obtain an almost sure bound for oscillation rates of empirical 
distribution functions for stationary causal processes. For short-range depen- 
dent processes, the oscillation rate is shown to be optimal in the sense that it is 
as sharp as the one obtained under independence. The dependence conditions 
are expressed in terms of physical dependence measures which are directly re- 
lated to the data-generating mechanism of the underlying processes and thus 
are easy to work with. 



1. Introduction 

Let E' ,Si,i € Z, be independent and identically distributed (iid) random variables 
on the same probability space (£l,A, P). For k S Z let 

(1-1) X k = g(...,E k -i,s k ), 

where g is a measurable function such that X k is a well-defined random variable. 
Then|Xfc}fe£z forms a stationary sequence. The framework is very general. 
See Hi, El El |23| among others. The process (X k ) is causal or non-anticipative 



in the sense that X k does not depend on future innovations £ k +i, £fc+2, ■ ■ •• Causal- 
ity is a reasonable assumption in practice. The Wiener-Rosenblatt conjecture says 
that, for every stationary and ergodic process X k , there exists a measurable func- 
tion g and iid innovations £j such that the distributional equality (X k )kei —v 



(g(. . . , £fc-i,£fc)}fcez holds; see [13|, l20(. For an overview of the Wiener-Rosenblatt 
conjecture see |9|. 

Let F be the cumulative distribution function of X k . Assume throughout the 
paper that F has a square integrable density / with square integrable derivative /'. 
In this paper we are interested in the oscillatory behavior of empirical distribution 
function 

1 n 

(1.2) F n (x) = - J2l Xi < x , xeR. 

n * — ' 

»=i 

In particular, we shall obtain an almost sure bound for the modulus of continuity 
for the function G n {x) = v / n[F rt (x) — F(x)]: 

(1-3) A„(6)= sup \G n (x) - G n (y)\, 

\x-y\<b 
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where b = b n is a sequence of positive numbers satisfying 
(1.4) b n — > and nb n — ► oo. 

Under the assumption that are iid, there exists a huge literature on the asymp- 
totic behavior of A„(-); see Chapter 14 in [l4| and the references cited therein. A 
powerful tool to deal with the empirical distribution F n is strong approximation 
[l|. In comparison, the behavior of A„(-) has been much less studied under depen- 
dence. In this paper we shall implement the new dependence measures proposed 
in [23l | and obtain an almost sure bound for A n . For a recent account of empirical 
processes for dependent random variables see the monograph edited by Dehling et 
al0. 

The rest of the paper is structured as follows. Main results on A n (6) are presented 
in Section [2] and proved in Section [3l Section [4] contains comparisons with results 
obtained under independence. Some open problems are also posed in Section 3J 

2. Main results 

We first introduce some notation. For a random variable Z write Z € C p (p > 0) if 
\\Z\\ p := (E|Z|P) 1 /*> < 00. Write || • || = || • || 2 . Let for fceZ, 

£fe = (• • • , ejfe-i, £fe) 

and for k > let be a coupled process of with £0 replaced by e' Q , i.e., 

£fc = (£-l,4> £ i; ■ ■ ■ i £ k)- 

We shall write X% = <?(££)■ Let k > 1 and define the conditional cumulative distri- 
bution function -Ffc(-|£o) by 

(2.1) F k (x\£ Q )=P(X k <x\Z ). 

Note that (Xi, £j) is stationary. Then for almost every £ G ■■■xlxl with respect 
to P we have 

(2.2) F k (x\z = = nx k < x\^ = = nx k+l < x\$i = 

for all i € Z. In other words, is the cumulative distribution function of the 

random variable £j+i, . . . , £»+&). Assume that for all fc > 1 and almost every 
£ 6 • • • x R x R with respect to P that F k (x\£,o = £) has a derivative /fc(x|£o = 
£), which is the conditional density of at x given £0 = £• By (12. 2[) . for any 
i£Z, /fe(^|£i) is the conditional density of X k +i at 2 given £j. Let the conditional 
characteristic function 

(2.3) y> fe (0|£o = = E(e^ eXfc |£ = = / e^"/ fc (t|$, = 

where \J — 1 is the imaginary unit. Our dependence condition is expressed in terms 
of the C 2 norm ||<p fc (0|fo) - !Pfc(0|£o)ll- 

Theorem 2.1. Assume that b n — > 0, logn = 0(nb n ) and that there exists a positive 
constant Cq for which 



(2.4) 



sup/i(x|£ ) < c 

X 
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holds almost surely. Further assume that 



E 

k=l 



1/2 



(2.5) J] I (i + e 2 )\\Mo\^)-Mmo)\\ 2 de • v. 

Let 
(2.6) 
Then 
(2.7) 

Roughly speaking, (12. 5| is a short-range dependence condition. Recall that 
fk{x\^o) is the conditional (predictive) density of Xk at x given £o and </?fc(0|<!;o) 
is the conditional characteristic function. So v?fc(0|£o) — <Pfc(0|£o) measures the de- 
gree of dependence of yfc(-|£o) on £o- Hence the summand in (|2.5|) quantifies a 
distance between the conditional distributions [Xfc|£o] and [^|Co] and (|2.5|) means 
that the cumulative contribution of eo in predicting future values is finite. 

For the two terms in the bound 



i(n) = (log n) 1 / 2 log log n. 



A„(6 n ) = O a . s .(V b n logn) + o a . s .[6„t(n)]. 



()2.7|) . the first one O a .s.(\/^n logn) has the same 
order of magnitude as the one that one can obtain under independence. See Chapter 
14 in 14j and Section 2J The second term o a . s . [6„t(n)] is due to the dependence of 

1/2 

the process (Xk). Clearly, if b n (loglogn) = o(l), then the first term dominates 
the bound in (|2.7p . The latter condition holds under mild conditions on b ni for 
example, if b n = 0(n~ r ') for some rj > 0. 

Let k > 1. Observe that <£ fe (0|£ o ) = E[<£i(0|£ fe _i)|£ o ] and 



(2.8) 



Efo^ifc-oie-i] =EM0i^_i)ie-i] =n<pi(m-i)\&] 



-i). Note that Si,e' , i S 
_ x ] = E[fc($U)l£-i] since ^ 



1 



are iid and 
1 is a coupled 



To see fl2J§, write h(&-i) = ¥>i(0|£fe 
k - 1 > 0. Then we have E[/i(£ fe _i)|f 
version of £fc_i with eo replaced by e' . On the other hand, we have E[/i(£J i] = 
E[h(£fc_l)|£o] since eo is independent of So (|2.8|) follows. Define the projection 
operator P/c by 

p fe z = E(zie fe )-iE(zia.-i), zee 1 . 

By the Jensen and the triangle inequalities, 



M0|£ o ) - M^o)\\ 



< 



iPk(e\^)-n^ k mo)\^-i}\\ 

+||E[^(0|£ o )|£-i] - 
= 2||P o ^(0|a-i)|| 
<2||^(0|6-i)-^(0|a-i)ll- 



Then a sufficient condition for (12.511 is 



(2.9) 



E 

k=0 



(l + 2 )|M0|£ fc )-^i(0|QH 2 ^ 



1/2 



< 00. 



In certain applications it is easier to work with (|2.9| . In Theorem 12.21 below we 
show that (|2.9p holds for processes (Xk) with the structure 



(2.10) 



Xk — £fc + Yk-it 
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where Y k -\ is = (..., e k -2, Sk-i) measurable. It is also a large class. The widely 
used linear process X k = a i £ k-i is of the form (12. 10|) . Nonlinear processes of 

the form 

(2.11) X k = m(X k _ x ) + e k 

also fall within the framework of (|2.10p if (|2.1ip has a stationary solution. A promi- 
nent example of (|2. 1 1[) is the threshold autoregressive model [19( 

X k = amax(X fc _i,0) + 6min(X fe _i, 0) + s k , 

where a and b are real parameters. For processes of the form (|2.10p . condition (|2.5p 
can be simplified. Let cp be the characteristic function of e\. 

Theorem 2.2. Let < a < 2. Assume (|2J0]) , 

(2.12) f |^(i)| 2 (l + t 2 )\t\ a dt < oo 
and 

oo 

(2.13) ^||X fe -^||«/ 2 <oo 

fc=0 

Then (|2.5|) is satisfied. 

Condition (|2.12p does not appear to be overly restrictive. It is satisfied if \<p{t)\ = 
0( 1 1 1 77 ) as \t\ — > oo, where rj > (3 + a)/2. It is also satisfied for symmetric-a- 
stable distributions, an important class of distributions with heavy tails. Let e k 
have standard symmetric a stable distribution with index < i < 2. Then its 
characteristic function ip(t) = exp(— \t\ L ) and (|2 . 1 2[) trivially holds. 

We now discuss Condition (|2. 13|) . Recall X'l = g(£ k )- Note that X^ and X k are 
identically distributed and X£ is a coupled version of X k with eo replaced by e' . If 
we view (11.11) as a physical system with £ k — (. . . , e k -i, £fc) being the input, g being 
a filter or tranform and Xi being the output, then the quantity X£ \\ a measures 
the degree of dependence of g(. . . , £ k -i, £ k ) on eo. In 2^] it is called the physical or 
functional dependence measure. With this input/output viewpoint, the condition 
(I2.13|) means that the cumulative impact of eo is finite, and hence suggesting short- 
range dependence. In many applications it is easily verifiable since it is directly 
related to the data-generating mechanism and since the calculation of \\X k — X k \\ a 
is generally easy [23j. In the special case of linear process X k = J^'jL o a j £ k~j with 
Cfc G C a and a = 2, then \\X k — X^\\ a — |afc||jeo — e || a and (|2.13[) is reduced to 
Xfclo ^ 00 ' w hi cri is a classical condition for linear processes to be short-range 
dependent. It is well-known that, if the latter condition is barely violated, then one 
enters the territory of long-range dependence. Consequently both the normalization 
and the bound in (|2.7j) will be different; see [a. l21|. 

For the nonlinear time series p. lip , assume that e k £ C a and p = sup x \m'(x)\ < 
1. Then (|2.1ip has a stationary distribution and \\X k — X k \\ a = 0(p k ) (see [24j). 
Hence (|2TT3"f holds. 

3. Proofs 

Lemma 3.1. Let H be a differential function on K. Then for any A > 0, 
(3.1) supiJ 2 (x) < A / H 2 {x)dx + \- 1 I [H'(x)] 2 dx. 

xGR JR JR 
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Proof. By the arithmetic mean geometric inequality inequality, for all x, y € K, 



H 2 (x)<H 2 (y) 



2H(t)H'(t)dt 



(3.2) <H 2 (y) + \ [ H 2 {x)dx + \- 1 [ [H'{x)] 2 dx. 

Jr Jr 

If infygR \H(y)\ > 0, then J M H 2 (x)dx = oo and (|3.1|) holds. If on the other hand 
inf y6 ffi \H(y)\ — 0, let (y n ) nG N be a sequence such that H(y n ) — > 0. So (|3.2|) entails 
(O). □ 



Lemma 13.11 is a special case of the Kolmogorov-type inequalities [18| . The result 
in the latter paper asserts that sup^gjn H 4 (x) < L H 2 {x)dx x J R H'(x) 2 dx. For the 
sake of completeness, we decide to state Lemma T3 . 1 1 with a simple proof here. 

Recall that |£o) is the conditional distribution function of Xk given £o (cf 
(|2.ip and (|2.2p ). Introduce the conditional empirical distribution function 

1 n i n 

i=l i=l 

Write 

(3.3) G n (x) = G° n (x)+G* n (x), 
where 

(3.4) G£(z) = yfr[F n (x) - F*(x)} and G*Jx) = ^K(i) - F(a:)]. 
Then 

n 

(3.5) VHG^(x) ^d,^) 

i=l 

is a martingale with respect to the filtration <t(£„) and the increments = 
^-Xi<x ~ E(lxi<x|Ci-i) are stationary, ergodic and bounded. On the other hand, if 
the conditional density /i(-|£i) exists, then G* is differentiable. The latter differen- 
tiability property is quite useful. 

Lemma 3.2. Recall (|2.6p for t(n). Lei = dG* n {x)/dx. Assume (|2.5p . TTien 

(3.6) sup|flr*(a:)| =o a . s .[t(n)]. 

Proof. Let fc > 1. Recall that /i(cc|£fc_i) is the one-step-ahead conditional density 
of Xk at x given £fc_i. By (|2.3j) . we have 



= / e^ xt Voh(x\^k-i)dt. 
Jr 

By Parseval's identity, we have 

|7>o^iW£fc-i)| 2 ^ = [ iVohixl^^dx 



and 



\VoMt\Sk-i)\ 2 t 2 dt = ±- I \Vof[(x\Ck-i)\ a dx. 

Z7T 
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Let 

a k = [ ||7Vi(z|6-i)H 2 ^ 
Jr 

and 

Jm. 

By ([211) and Jensen's inequality, ||P o <Pi(0|&-i)|| < ||Wfc(0|£o) - <Pk(P\to)\\- So 423 
implies that 



(3.7) J2^ ak+I3k <oc - 

fc=i 

Let A = X^feLi V^fe an( i for fc > 

k 



i=i 



Then Hfc(a:) = \fkg k (x). Note that for fixed I e N, 7V;/i(2|6-i), i = 1,2,..., 
are stationary martingale differences with respect to the filtration cr(^_;). By 
the Cauchy-Schwarz inequality and Doob's maximal inequality, since Hk(x) = 

E / max^(,)& < E / £ maXfc ^" ■ ^ 7Wi(^-i)| 2 Ada; 



2 



<^4n||Po/^-r)|| 2 A ^ = 4nA2 = 0(n) 



1=1 v ai 
Similarly, since J2k=i Vflk < oo, 

E / max|^(a;)| 2 dx = O(n). 

By Lemma 13-11 with A = 1, we have 

E[max fe < 2 d sup xm |fl" fc (a;)| 2 ] 



=i 

oo 

<£ 

oo 
oo 



2 d i 2 (2 d ) 

E[max fc < 2 , jg |ff fc (:r)| 2 + |g^(o;)| 2 da;] 
2 rf i 2 (2 d ) 

E J K max fc < 2 d |Hfc(a;)| 2 + max fc < 2 <i |.ffj(,(a;)| 2 da; 



2<V(2 d ) 



v ^ Q(2 d ) 

jL 2 *t 2 (2«*) <0 °- 



By the Borel-Cantelli lemma, sup^ max fc<2 d |J?fc(x)| = o a . s . [2 d / 2 i(2 d )] asd— > oo. For 
any n > 2 there is a e? € N such that 2 d ~ 1 < n < 2 d . Note that maxfe< n |JTfc(x)| < 
max fe < 2 ti and t(n) is slowly varying. So (|3 - 6[) follows. □ 
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Lemma 3.3. Assume logn = 0{nb n ) and Xk € C a for some a > 0. Then for any 
t > 2, there exists C = C T > such that 



(3.8) 



sup \G> n (x)-G* n {y)\>Cy/K]ogn 

\x-y\<b n 



0(n~ T ). 



Proof. We can adopt the argument of Lemma 5 in 22|. Let Xq = n^ 3+r ^ a . Then 
for any C > 0, by Markov's inequality, 



(3.9) 



F[n{F n (-x ) + 1 - F n (x )} > cyVogn] 
nE{F n {-x ) + 1 - F n (x )} 



< 



< 



C^Jb„ logn 
na;o Q E(|X | Q 



CV&nlogn 



(9(n- T ). 



It is easily seen that the preceding inequality also holds if F n is replaced by F* . 

Recall (13.51) for G^(x) and di(x) = lxi<x — E(lx 4 <s|&-i)- Let cc < y < x + b n 
By (EH), 



x<Xi<y\£i-l) < nb n c Q . 



By Freedman's inequality in [6|, if \x — y\ < we have 



V"|G^(a;) - Gn(y)\ > C \J nb n log n 



< 2exp 



-C 2 nb n logn 



C\]nb n logn + n&„c 



Let 6„ = {— x + k/n 3 : k = 0,..., [2a:o^ 3 J}- Since logn = 0(nb n ), it is easily 
seen that there exists a. C — C T such that 



(3.10) 



sup y/n\G*(x) ~ G° n (y)\ > C^fnb n logn 

x,y^Q n .\x—y\<.b n 

—C 2 nb n logn 



= O(x n ) exp 



C\Jnb n logn + nb n co 



= 0(n- T ). 



For every a; G [—xq, Xq], there exists a 9 £ 9„ such that 9 < x < 9+1/n 3 . So (13. 10[) 
implies that 



(3.11) 



sup MG n (x) - G° n (y)\ >(C + l)^nb n logn 

\x\<x ,\y\<x ,\x-y\<b 71 . 



= 0{n- T ) 

in view of the monotonicity of F n {-) and the fact that, if 9 < <j) < 9 + 1/n 3 , 



^ E(l e <Xi<0|Ci-l) <\4>~ 8\nco = c-o/n 2 = o{yJ nb n logn). 



Combining ([53]) and ([3TTT]) . we have ([3~%]) . □ 
Proof of Theorem \2.1\ By (|3.3p , it easily follows from Lemmas 13.21 and 13.31 □ 
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Proof of Theorem \2.SX By Lemma 13.11 and Parseval's identity, (|2. 12[) implies 

sup f{x) < J {f{x) + [f'(x)] 2 }dx = i- J \ V (t)\ 2 {l + t 2 )dt < oo. 

Since Sk and Yfc_i are independent, 

E[e^ =TtXk \Ck-i] = e^ =ItY ^ i p(t). 

Note that 

|| e V=TO^-i^) _ e ^*i7-i^(t)||2 

= \ V (t)\ 2 \\e^~ UY ^ - e^~ ltY ^\\ 2 

<A\ V (t)\ 2 \\mm{l,\tY k ^-tYU\)\\ 2 
<4\ip(t)\ 2 E(\tY k ^-tY k U\ a )- 

Hence <(2l5j) follows from (|27[3|) . □ 



4. Conclusion and open problems 

Let Xi be iid standard uniform random variables. Stute [16] obtained the following 
interesting result. Assume that b n — > is a sequence of positive numbers such that 

(4.1) logn = o(nb n ) and loglogn = o(log&~ 1 ). 
Then the convergence result holds: 

(4.2) lim _ ^ a i most sure iy. 
™^°° \/& n log fen 1 

If there exists rj > such that 6 n + (n&„) _1 = 0(n~ ,? ), then the bound in (|2.7p 
becomes \/6 n log n, which has the same order of magnitude as \J b n log bn 1 , the 
bound asserted by (|4.2p . It is unclear whether there exists an almost sure limit for 

A„ (&„ ) / x/bn log 1 if the dependence among observations is allowed. Mason et al 
[lH considered almost sure limit for A„ (&„)/•*/&„ log fr^ 1 when (|4.ip is violated. 
Deheuvels and Mason 0] (see also 0) proved functional laws of the iterated loga- 
rithm for the increments of empirical processes. Local empirical processes in high 
dimensions have been studied in [B|, 0, [13] ■ It is an open problem whether similar 
results hold for stationary causal processes. We expect that our decomposition (|3.4p 
will be useful in establishing comparable results. 
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